1. Field of the Invention
The present invention relates to a scanning line interpolating apparatus for forming scanning lines having no input signals thereon by interpolation on the basis of adjacent scanning lines to convert a format of motion picture signals. Further, this invention relates to a motion vector detecting apparatus for motion compensation interpolation.
2. Description of the Prior Art
The standard television signals such as NTSC (National Television System Committee) and Hi-Vision signals are of interlaced signals; that is, as shown in FIG. 1A, one frame is composed of two fields shifted in time and the vertical direction.
In contrast with this, the scanning line structure having no time shift, as shown in FIG. 1B, is referred to as non-interlaced scanning or progressive scanning.
The interlaced signals generate flickers when the high frequency components of the video signals increase in the picture vertical direction.
To overcome this problem, there exists such a processing that scanning lines not existing between the two adjacent interlaced scanning lines are formed by interpolation on the basis of the adjacent scanning lines, as shown in FIG. 1C. The processing as described above is referred to as progressive scanning conversion or double-density conversion.
In this case, the scanning lines are interpolated in accordance with a motion-adaptable processing. In more detail, when the picture is moving, the scanning lines are formed by interpolation on the basis of the vertically adjacent scanning lines on the same field, as shown in FIG. 2A. When the picture is still, however, the scanning lines are formed by interpolation on the basis of the scanning lines located at the same positions of two before and after fields different with respect to time, as shown in FIG. 2B.
In addition, recently, a method of applying motion compensation to the inter-field interpolation has been studied. In this case, the scanning lines are formed by interpolation on the basis of the scanning lines located at different positions of two before and after fields with respect to time, as shown in FIG. 2C.
On the other hand, in the standard of the high efficiency coding methods such as H. 261 of ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) or MPEG I (Moving Picture Experts Group 1) of ISO/IEC (International Organization for Standardization)/(International electrotechnical Commission), the non-interlaced video signals are to be coded. Therefore, when the interlaced video signals are used as the signal sources, it is necessary to first convert the interlaced video signals into non-interlaced video signals.
In the above-mentioned standard, since the number of pixels to be coded is smaller than that of the ordinary TV signals, there arises no problem when interlaced video signals of any one of the fields are used as the coded video signals. However, in the case of the simple reduction of the interlaced video signals, since the amount of aliasing increases, a problem arises in that the picture quality (from a subjective point of view) and the coding efficiency both deteriorate.
To eliminate the aliasing, it is necessary to form frames to be scanned by the progressing scanning and to filter the formed frames in the vertical direction through an appropriate filter to reduce the scanning lines. In this case, however, an ideal processing cannot be executed when the sequential frames are not formed appropriately.
A conventional scanning line interpolating apparatus using the motion compensation will be explained hereinbelow with reference to FIG. 3. This apparatus is disclosed in [Study of a progressive scanning conversion method for interlaced picture using motion compensation and its apparatus], Institute of Television Engineers of Japan, Technical Report, BCS 93-70.
In FIG. 3, interlaced video signals inputted through a video signal input 1 are applied to a field delay circuit 2, a motion compensator 3, and a motion vector (referred to as MV, hereinafter) detector 20, respectively.
In the field delay circuit 2, the signals are delayed by a period of time corresponding to one field, and the output signals are applied to an intraframe interpolator 9 and a field delay circuit 15.
The field delay circuit 15 delays the signals by a period of time corresponding to one field in the same way as in the field delay circuit 2, and the output signals are applied to another motion compensator 16.
Therefore, three video signals being delayed by different number of fields are applied to the motion compensator 3, the intraframe interpolator 9 and the other motion compensator 16.
On the other hand, the MV detector 20B obtains a motion vector of video signals between two fields, and the obtained values are applied to the two motion compensators 3 and 16. In accordance with the motion vector values, the motion compensators 3 and 16 shift the input video signals spatially, and output the shifted video signals.
Here, since the time relationship is opposite between the fields motion-compensated by the motion compensators 3 and 16, from the interpolated field's point of view, the shift directions are opposite to each other, as shown in FIG. 2C.
The video signals whose motion is compensated as described above are applied from the motion compensators 3 and 16 to an adder 4 and a subtracter 11.
The adder 4 adds two motion-compensated field signals with two-field difference, and multiplies the added field signals by 1/2 as the interframe interpolation signal. The interframe interpolation signal is applied to a multiplier 6.
The subtracter 11 obtains a difference between the two motion-compensated field signals with two-field difference, and the difference signal is applied to an absolute value converter 17 to obtain an absolute value of the difference signal. The absolute difference signal is applied to a spatial LPF (low-pass filter) 19 to smoothen the spacial variation of the absolute difference signal. The smoothened absolute difference signal is applied to a non-linear converter 14.
The non-linear converter 14 converts the output of the space LPF 19 non-linearly into a value "k" indicative of the matching rate between pictures.
The conversion characteristics are determined as 0 when the output level of the spatial LPF 19 is less than a noise level, and as 1 when the intraframe interpolation level is clearly higher than the interframe interpolation level by an appropriate value. Further, the conversion characteristics are linear between "0" and "1". The value k thus obtained is applied to the multiplier 6 and another multiplier 10.
On the other hand, the intraframe interpolator 9 adds two video signals on upper and lower scanning lines to be interpolated, as shown in FIG. 2A, to form an intraframe interpolated scanning line. Here, the delay generated by the motion compensation can be compensated, and the intraframe interpolation signal is applied to the multiplier 10 in synchronism with the interframe interpolation signal.
To the two multipliers 6 and 10, the value k indicative of the matching rate is given from the non-linear converter 14. The multiplier 6 multiplies the interframe interpolation signals by (1-k), and the multiplier 10 multiplies the intraframe interpolation signal by k. The multiplied results are applied to an adder 7.
The adder 7 adds the interframe interpolation signal (x (1-k)) and the intraframe interpolation signal (x k) both weighted by the matching rate to obtain a final interpolation signal. The final interpolation signal is outputted through an interpolation signal output 8.
To generate the sequential scanning signals on the basis of the above-mentioned interpolation signal, a sequential scanning converting apparatus as shown in FIG. 4 is used.
In the sequential scanning converting apparatus show in FIG. 4, interpolation signals outputted by a scanning line interpolating apparatus 50 are applied to a line buffer 52. The video signals are delayed by a field delay circuit 2 to compensate the processing delay caused by the scanning line interpolating apparatus 50, and then applied to a line buffer 51. The two line buffers 51 and 52 hold video signals for one line. The video signals held by the buffer 51 or 52 are read at a speed twice higher than the input signals. These read signals are selected alternately through a switch 53 as sequential scanning line signals, and then outputted through a video output 54.
A motion vector detecting apparatus (which corresponds to the MV detector 20B shown in FIG. 3) for interpolating the motion compensation scanning lines will be explained with reference to FIG. 5.
In FIG. 5, interlaced scanning video signals inputted through a video input 1 are applied to a field delay circuit 2, a motion compensator 3, and a temporal MV generator 21.
The field delay circuit 2 and another field delay circuit 15 delay video signals by one field. Therefore, the video signals delayed by one frame in total are applied to a motion compensator 16.
The temporal MV generator 21 generates motion vector values (MVs) in sequence in a predetermined MV search range. For instance, when the search range is determined as .+-.3 in the vertical direction in scanning line unit and .+-.7 in the horizontal direction in pixel unit, MV values of 105=7.times.15 (vertical x horizontal) are generated in sequence. The generated MV values are applied to the two motion compensators 3 and 16.
In accordance with the motion vector values, the two motion compensators 3 and 16 shift the input video signals spatially, and then output shifted video signals.
The motion compensating processing is executed in block unit (e.g., 16.times.8 pixels), and the MV is one value in the block.
Further, the field compensated by the motion compensator 3 and the field compensated by the motion compensator 16 are opposite to each other in time relationship from the interpolated field's point of view, so that the shift directions are opposite to each other as shown in FIG. 2C.
The video signals thus motion-compensated are applied to a subtracter 11 to obtain interframe difference signals. The difference signals are given to an absolute value converter 17.
The absolute value converter 17 obtains the absolute difference signals (e.g, by squaring the difference signals). The obtained absolute difference signals are applied to a block accumulator 24.
The block accumulator 24 accumulates the absolute difference signals for one block as a value indicative of matching rate, and the obtained value is applied to a MV selector 22. More specifically, the MV values and the matching values between two frames motion-compensated on the basis of the MV values are both inputted to the MV selector 22.
The MV selector 22 compares the MV values and the matching values to select MV values of the best matching (in which the accumulated value of the difference signals is the minimum). The selected MV values are outputted as the final MV values through an MV output 23.
Although the operation of the motion compensators 3 and 16, the subtracter 11, and the absolute value converter 17 are the same as with the case of those shown in FIG. 3, the same processing is executed for each MV, the quantity of processing increases in proportion to the number of MVs.
In other words, in order to process the motion picture in real time, since a high processing speed is required in proportion to the number of MVs, the parallel processing may be required at need.
In the conventional scanning line interpolating apparatus, the interframe interpolation and the intraframe interpolation are switched and further the motion vectors are selected for motion-compensated scanning line interpolation both on the basis of the interframe matching between the fields before and after a field to be interpolated. Therefore, as far as the interframe matching is excellent, the picture in which the interpolation signals are quite different from the upper and lower scanning lines is selected.
In particular, when the motion compensation is executed, since the video signals positioning away from each other spatially are used, there exists a problem in that inappropriate interpolation is executed.